Skip to content

Comments

[8.x](backport #41477) [AWS] [EC2] enrich events with EC2 tags with add_cloud_metadata processor#41615

Merged
Kavindu-Dodan merged 1 commit into8.xfrom
mergify/bp/8.x/pr-41477
Nov 12, 2024
Merged

[8.x](backport #41477) [AWS] [EC2] enrich events with EC2 tags with add_cloud_metadata processor#41615
Kavindu-Dodan merged 1 commit into8.xfrom
mergify/bp/8.x/pr-41477

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented Nov 12, 2024

Proposed commit message

This PR adds support to enrich events with EC2 tags iff,

  • IMDS endpoint is enabled 1
  • Tag access is enabled through IMDS endpoint. 2

Tags are added to event payload with key aws.tags.<KEY>,

{
  "aws" : {
   "tags" : {
      "org" : "acme",
      "owner" : "userA"
    }
   }
}

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

The best way to validate this is through a beats deployment (ex:- metricbeat) in an EC2 instance.

  • Create an EC2 instance
    • Enable IMDS endpoint
    • Enable tags access through IMDS endpoint
    • Assign instance with a role with adequate permissions
    • Add few tags to the instance
  • Build metricbeat from this branch
  • Deploy custom build into EC2
  • Enable add_cloud_metadata processor
  • Run and observe metrics containing tags

Related issues

Closes #31899

Screenshots

image

Footnotes

  1. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-options.html

  2. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/work-with-tags-in-IMDS.html#allow-access-to-tags-in-IMDS


    This is an automatic backport of pull request [AWS] [EC2] enrich events with EC2 tags with add_cloud_metadata processor #41477 done by Mergify.

…ssor (#41477)

* add support to extract ec2 tags from IMDS endpoint

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

* add dedicated tests for tag extractor

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

* expand test case and add documentation

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

* add changelog entry

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

* handle empty tags, add tests and close underlying body

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

* review change - use aws.tags as tag prefix

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

---------

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>
(cherry picked from commit c878397)
@mergify mergify bot requested a review from a team as a code owner November 12, 2024 15:13
@mergify mergify bot added the backport label Nov 12, 2024
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Nov 12, 2024
@Kavindu-Dodan Kavindu-Dodan enabled auto-merge (squash) November 12, 2024 15:18
@Kavindu-Dodan Kavindu-Dodan added the Team:obs-ds-hosted-services Label for the Observability Hosted Services team label Nov 12, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport Team:obs-ds-hosted-services Label for the Observability Hosted Services team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants